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Question 2 


Intent of Question 





The primary goals of this question were to assess students’ ability to (1) distinguish an experiment from an 
observational study; (2) critique statistical information, in particular whether or not researchers are justified 
in making a specific conclusion based on the given information; (3) recognize and describe a potential 
problem with a study that lacks random assignment or blinding. 


Solution 
Part (a): 


The study was an experiment because treatments (D-cycloserine or placebo) were imposed by the 
researchers on the people with acrophobia. 


Part (b): 


No, the experiment was designed to compare the D-cycloserine group with a control group that 
received the placebo. The researchers can conclude that the D-cycloserine pill and two therapy 
sessions show significantly more improvement than a placebo and two therapy sessions. However, 
there is no basis for comparison with another group of people with acrophobia who received eight 
therapy sessions and no pill. 


Part (c): 


One example is that if the therapists were allowed to choose who received the placebo and who 
received D-cycloserine, they might assign the people with more severe acrophobia to one of the groups 
and the people with less severe acrophobia to the other group. Thus, the improvement after only two 
therapy sessions could be related to the initial severity of the acrophobia rather than to the effects of 
D-cycloserine. 


Scoring 


Parts (a) and (c) are scored as essentially correct (E), partially correct (P), or incorrect (I). Part (b) is scored as 
essentially correct (E) or incorrect (I). 


Part (a) is scored as follows: 


Essentially correct (E) if the response indicates that this was an experiment, AND the explanation 
clearly communicates that two treatments were imposed. 


Partially correct (P) if the response indicates that this was an experiment, BUT the explanation does 
not clearly communicate that two treatments were imposed. 


Note: If the response indicates that this was an experiment because there was random assignment to 
treatments, this implies imposition of treatments and is scored as E. If the response does not clearly 
state that the random assignment is to treatments, this is scored as P. 


Incorrect (I) if the response indicates that this is an observational study OR if the explanation is 
missing or incorrect. 


© 2011 The College Board. 
Visit the College Board on the Web: www.collegeboard.org. 


AP® STATISTICS 
2011 SCORING GUIDELINES (Form B) 


Question 2 (continued) 
Part (b) is scored as follows: 


Essentially correct (E) if the response says “no” AND clearly explains why this is not reasonable based 
on the fact that there was no experimental group that received eight therapy sessions and no pill. 


Incorrect (I) if the response provides an answer with an incorrect or no justification. 
Part (c) is scored as follows: 


Essentially correct (E) if the response indicates that this method of assignment might create 
experimental groups that differ in some systematic way other than the treatment AND provides a 
justification that describes the potential confounding, 

OR 
if the response indicates that if the therapists know who was in which group, it may influence the 
therapists’ behavior when dealing with or evaluating the people with acrophobia. 


Partially correct (P) for any of the following: 


e The response indicates that the assignment might create experimental groups that differ in 
some systematic way, BUT does not provide an explanation of the potential confounding. 


= 


e The response makes a general statement that the lack of random assignment could lead to 
confounding BUT does not provide an example in context. 

e The response indicates that the therapists know who is in which group BUT does not give a 
reason as to why this might lead to a misleading conclusion. 


e The response makes a general statement that failure to blind may lead to bias BUT does not 
provide an example in context. 





Incorrect (I) if the response fails to meet the criteria for E or P. 
Note: If the response discusses incorrect conclusions that might result from having the people with 
acrophobia (rather than the therapists) choose their own treatments, the response is scored as I, 
because such a response does not address the question asked. 
4 Complete Response 
All three parts essentially correct 
3 Substantial Response 
Two parts essentially correct and one part partially correct 
2 Developing Response 
Two parts essentially correct and one part incorrect 


OR 
One part essentially correct and one or two parts partially correct. 
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Question 2 (continued) 
Minimal Response 
One part essentially correct and two parts incorrect 


OR 
One or two parts partially correct and one part incorrect 
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2. People with acrophobia (fear of heights) sometimes enroll in therapy sessions to help them overcome this fear. 
Typically, seven or eight therapy sessions are needed before improvement is noticed. A study was conducted to 
determine whether the drug D-cycloserine, used in combination with fewer therapy sessions, would help people 
with acrophobia overcome this fear. 


Each of 27 people who participated in the study received a pill before each of two therapy sessions. Seventeen 

of the 27 people were randomly assigned to receive a D-cycloserine pill, and the remaining 10 people received a 
placebo. After the two therapy sessions, none of the 27 people received additional pills or therapy. Three months 
after the administration of the pills and the two therapy sessions, each of the 27 people was evaluated to see ifhe 
or she had improved. 


(a) Was this study an experiment or an observational study? Provide an explanation to support your answer. 
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(b) When the data were analyzed, the D-cycloserine group showed statistically significantly more improvement 
than the placebo group did. Based on this result, would the researchers be justified in concluding that the 
D-cycloserine pill and two therapy sessions are as beneficial as eight therapy sessions without the pill? 
Justify your answer. 
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(c) A newspaper article that summarized the results of this study did not explain how it was determined which 
people received D-cycloserine and which received the placebo. Suppose the researchers allowed the 
therapists to choose which people received D-cycloserine and which received the placebo, and no 
randomization was used. Explain why such a method of assignment might lead to an incorrect conclusion. 
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2. People with acrophobia (fear of heights) sometimes enroll in therapy sessions to help them overcome this fear. 


Typically, seven or eight therapy sessions are needed before improvement is noticed. A study was conducted to 
determine whether the drug D-cycloserine, used in combination with fewer therapy sessions, would help people 
with acrophobia overcome this fear. 


Each of 27 people who participated in the study received a pill before each of two therapy sessions. Seventeen 
of the 27 people were randomly assigned to receive a D-cycloserine pill, and the remaining 10 people received a 
placebo. After the two therapy sessions, none of the 27 people received additional pills or therapy. Three months 
after the administration of the pills and the two therapy sessions, each of the 27 people was evaluated to see if he 
or she had improved. 


(a) Was this study an experiment or an observational study? Provide an explanation to support your answer. 
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(b) When the data were analyzed, the D-cycloserine group showed statistically significantly more improvement 
than the placebo group did. Based on this result, would the researchers be justified in con¢luding that the 
D-cycloserine pill and two therapy sessions are as beneficial as eight therapy sessions without the pill? 
Justify your answer. 
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(c) A newspaper article that summarized the results of this study did not explain how it was determined which 
people received D-cycloserine and which received the placebo. Suppose the researchers allowed the 
therapists to choose which people received D-cycloserine and which received the placebo, and no 
randomization was used. Explain why such a method of assignment might lead to an incorrect conclusion. 
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GO ON TO THE NEXT PAGE. 
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2. People with acrophobia (fear of heights) sometimes enroll in therapy sessions to help them overcome this fear. 
Typically, seven or eight therapy sessions are needed before improvement is noticed. A study was conducted to 
determine whether the drug D-cycloserine, used in combination with fewer therapy sessions, would help people 
with acrophobia overcome this fear. 


Each of 27 people who participated in the study received a pill before each of two therapy sessions. Seventeen 
of the 27 people were randomly assigned to receive a D-cycloserine pill, and the remaining 10 people received a 
placebo. After the two therapy sessions, none of the 27 people received additional pills or therapy. Three months 
after the administration of the pills and the two therapy sessions, each of the 27 people was evaluated to see if he 
or she had improved. 


(a) Was this study an experiment or an observational study? Provide an explanation to support your answer. 
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(b) When the data were analyzed, the D-cycloserine group showed statistically significantly more improvement 
than the placebo group did. Based on this result, would the researchers be justified in concluding that the 
D-cycloserine pill and two therapy sessions are as beneficial as eight therapy sessions without the pill? 
Justify your answer. 
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(c) A newspaper article that summarized the results of this study did not explain how it was determined which 
people received D-cycloserine and which received the placebo. Suppose the researchers allowed the 
therapists to choose which people received D-cycloserine and which received the placebo, and no 
randomization was used. Explain why such a method of assignment might lead to an incorrect conclusion. 
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Question 2 


Sample: 2A 
Score: 4 


In part (a) the response indicates that subjects were assigned treatments by the researchers and that there 
was more than one treatment. Part (a) was scored as essentially correct. Part (b) is answered concisely and 
correctly by noting that the eight therapy sessions without the pill was not one of the treatments included 
in the study. Part (b) was scored as essentially correct. The first sentence in part (c) correctly indicates that 
allowing the therapists to assign people to groups might result in experimental groups that differ in “fear 
intensity,” but it is not accompanied by an explanation of how this potential confounding could lead to an 
incorrect conclusion. However, the second and third sentences in part (c) indicate that allowing the 
therapists to assign people to groups would compromise double-blinding, with the consequence that the 
therapists might “adjust the therapy accordingly.” The statement that the therapists’ behavior when 
dealing with the people with acrophobia might be influenced by their knowledge of treatment 
assignments was considered adequate explanation of why an incorrect conclusion might be reached, and 
part (c) was scored as essentially correct. Because three parts were scored as essentially correct, the 
response earned a score of 4. 


Sample: 2B 
Score: 3 


In part (a) the response indicates correctly that the study was an experiment because the independent 
variable was “manipulated” and the dependent variable was “to be observed.” The response also indicates 
that there was a “control group and experimental group.” Part (a) was scored as essentially correct. Part (b) 
was answered correctly by noting that because the study includes only treatments with two therapy 
sessions, conclusions can be made only about treatments with two therapy sessions. Part (b) was scored 
as essentially correct. The response in part (c) indicates that lack of random assignment could lead to 
confounding, but no example in the context of the study is provided. Part (c) was scored as partially 
correct. Because two parts were scored as essentially correct and one part was scored as partially correct, 
the response earned a score of 3. 


Sample: 2C 
Score: 2 


The response in part (a) indicates that the study was an experiment “because the researcher directly 
introduced a variable (the drug) to see its effects on the subjects.” Although this response suggests 
imposition of a treatment, it does not clearly communicate that two treatments were imposed, and 

part (a) was scored as partially correct. In part (b) the response incorrectly states that the researchers would 
be justified in reaching a conclusion about eight therapy sessions, so part (b) was scored as incorrect. The 
response in part (c) indicates that if therapists are permitted to assign the people with acrophobia to 
treatment groups, they might assign those with less fear to the D-cycloserine group, which could result in 
overestimation of the effectiveness of the drug. This is a clear explanation of potential confounding, and 
part (c) was scored as essentially correct. Because one part was scored as essentially correct, one part was 
scored as partially correct, and one part was scored as incorrect, the response earned a score of 2. 
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